====== Configuration Server ======

Setup of a survey server based on a Debian or Ubuntu server.


===== Aditional for www.soscisurvey.de =====

  * OwnCloud
  * Shop/Lizenzserver
  * Softwarelizenzes ''/download''
  * DokuWiki manual
  * Homepage

Firewall: Outgoing ports 443 (license server) and 25 (SMTP for sending mail, if directly from server)

===== Additional for own Server =====

  * Additional user ''s2access'' gets a 32-digit password, login via password possible
  * Possibly the disk still needs to use the memory space first:\\ ''lvextend -r -l +100%FREE /dev/ubuntu-vg/ubuntu-lv''

==== Owncloud ====

<code bash>
sudo vim /etc/logrotate.d/owncloud
</code>

<file conf /etc/logrotate.d/owncloud>
/var/www/s2survey/html/owc/data/owncloud.log
{
    weekly
    dateext
    rotate 10
    compress
    delaycompress
    missingok
    notifempty
    su www-data www-data
    create 0640 www-data www-data
    sharedscripts
}
</file>


===== Preparation =====

==== Clarify Configuration ====

  * How do I get the SSH access data for the server?
  * Should SSH login be restricted to login via keyfile?
  * Should the server be set up via IPv4 or IPv6 only or via DualStack?
  * Is a firewall available and how is port 80 (HTTP) and 443 (HTTPS) and port 22 (SSH) enabled for a defined IP range, if necessary?
  * What is the e-mail address to which server status messages and user requests should be sent?
  * We need a mailbox for e-mail returns incl. access data so that the server can read the returns.
  * Is the mail sent directly through the server or via a relay server?
    * Which relay server is used for sending mails (for serial mails and notifications) and which limits are set there?
    * Is port 25 for outgoing e-mails enabled in the firewall?
  * What is the (sub)domain through which the survey server is accessible? Do you already have a DNS record (A and/or AAAA) and a PRT for it?
  * How to apply for the SSL certificate?
  * How is the server backed up? Where can data be backed up off-location if the server is not automatically backed up by the data center?
  * Should only the database (collected data) be encrypted or also the files (relevant if respondents upload e.g. pictures)? In the second case, the data center would have to set up an encrypted partition/container and ensure that it is automatically mounted at boot time.


==== Prepare Keywords in KeePass ====

  * SSH-User
  * SSH Keyfile sosci (PuttyGen)
  * SSH Keyfile www-ftp (evtl.)
  * SSH Keyfile for backup
  * MySQL
    * master
    * s2survey
    * backup
  * Monit (admin)
  * Munin (admin)
  * Unix-User: mailer (if local mailbox planned)
  * SoSci-PW: admin
  * SoSci-PW: Cronjobs

View changed configuration files (in case of an update).


<code>
sudo apt install debsums
sudo debsums -ce
diff /etc/whatever{,.dpkg-dist}
</code>

  * Note access data in KeePass
    * IP adress
    * URL
    * Username/Keyword
    * MySQL-Port (Weiterleitung)
  * Create Putty entry anlegen
  * Key-Note fingerprints for SSH (''ssh_host_rsa_key'' and all others, sometimes as md5, sometimes without):

<code bash>
sudo ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key -E md5
ssh -o FingerprintHash=md5 localhost
ssh -o FingerprintHash=sha256 localhost
</code>

Creating a key for ''dhparam'' takes a while.

<code bash>
sudo mkdir /var/www/
sudo mkdir /var/www/ssl/
sudo openssl dhparam -out /var/www/ssl/dhparam.pem 4096
</code>

Then continue working in a new Putty instance.


===== Create Users (if needed) =====

<code>
sudo useradd -m sosci
sudo useradd -m s2access
sudo passwd sosci
sudo passwd s2access
sudo usermod -a -G sudo sosci
sudo usermod -a -G sudo s2access
</code>

Still change the shell:

<code>
sudo vim /etc/passwd
sosci:x:1001:1001::/home/sosci:/bin/bash
s2access:x:1002:1002::/home/s2access:/bin/bash
</code>


===== SSH Setup =====

SSH KeepAlive, see [[https://patrickmn.com/aside/how-to-keep-alive-ssh-sessions/|How to Keep Alive SSH Sessions]]

<code bash>
sudo vim /etc/ssh/ssh_config

Host *
    ServerAliveInterval 300
    ServerAliveCountMax 2
</code>

<code bash>
sudo vim /etc/ssh/sshd_config

# KeepAlive intervals
ClientAliveInterval 60
ClientAliveCountMax 2
</code>

No keyword for ''root''

<code>
sudo vim /etc/shadow

root:*:....
</code>
  
Access via keyfile ([[https://vorkbaard.nl/using-putty-and-keyfiles-to-ssh-into-your-ubuntu-12-04-server/|Using PuTTY and keyfiles to SSH...]]), but login as ''sosci''.

<code>
mkdir ~/.ssh
chmod -R 700 ~/.ssh
vim ~/.ssh/authorized_keys
# Inhalt aus ''puttygen''
chmod 600 ~/.ssh/authorized_keys

# Edit /etc/ssh/sshd_config so it contains
sudo vim /etc/ssh/sshd_config

AuthorizedKeysFile %h/.ssh/authorized_keys
PermitRootLogin no

sudo service ssh restart
</code>

Disable password login for everything except s2access, cf. [[https://serverfault.com/q/285800/100194|How to disable SSH login with password for some users?]]

<code>
sudo vim /etc/ssh/sshd_config
# Insert at the end of the file

# Allow login only via Keyfile (with exceptions)
PasswordAuthentication no
Match User s2access
PasswordAuthentication yes

sudo service ssh restart
</code>

Test login via KeyFile (new Putty instance)

Delete any previous user.

<code>
sudo deluser whatever
</code>


===== Software =====

  * Some small utilities
    * Browser ''elinks'' to retrieve the server status
  * Some security features
    * [[http://munin-monitoring.org/wiki/munin.conf|Munin]] for monitoring the server performance -- [[http://www.darkcoding.net/software/setting-up-munin-on-ubuntu/|Setup-Tips]]
    * Monit for monitoring the accessibility (incl. restarting the server in case of problems)
    * iptables for firewall ([[http://www.debian-administration.org/articles/445|Getting IPTables to survive a reboot]])
  * Basis-Software for webserver
    * MySQL database
    * Apache Webserver ([[http://library.linode.com/web-servers/apache/installation/debian-6-squeeze|Apache 2 Web Server on Debian 6]]) or ''nginx''
    * ''apachetop'' for quick control of the server load
    * PHP
    * Certbot for domain and HTTPS connection via Let's Encrypt

==== nginx-Package sources ====

Source: http://nginx.org/en/linux_packages.html

<code bash>
# Install the prerequisites:
sudo apt install curl gnupg2 ca-certificates lsb-release

# Set up the apt repository for stable nginx packages
echo "deb http://nginx.org/packages/ubuntu `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list

# Set up repository pinning to prefer our packages over distribution-provided ones:
echo -e "Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n" \
    | sudo tee /etc/apt/preferences.d/99nginx

# import an official nginx signing key. Fetch the key:
curl -o /tmp/nginx_signing.key https://nginx.org/keys/nginx_signing.key

# Verify that the downloaded file contains the proper key
# The output should contain the full fingerprint 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62
gpg --dry-run --quiet --import --import-options show-only /tmp/nginx_signing.key

#Finally, move the key to apt trusted key storage (note the "asc" file extension change)
sudo mv /tmp/nginx_signing.key /etc/apt/trusted.gpg.d/nginx_signing.asc
</code>


==== MariaDB Backup ====

Not used so far, because the incremental backups with the deletion periods are not quite trivial. And the configuration of one new incremental set per month I haven't tried/managed yet.

<code bash>
curl -LsS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash
sudo apt update
sudo apt dist-upgrade
sudo apt install mariadb-backup
</code>

Possibly ''apt'' is acting up because the dependencies for MariaDB backup don't work. In that case install ''aptitude'' and try the installation again. It may be necessary to install a newer version of MariaDB than is available by default through the distribution.


==== Installation Packages ====

<code>
# update system 
sudo apt update
sudo apt dist-upgrade
sudo apt remove apache2
sudo apt autoremove

# install Software
sudo apt install less vim elinks debsums fail2ban munin libwww-perl monit unattended-upgrades duplicity duply tmux ncdu
sudo apt install iptables rkhunter chkrootkit mailutils apt-transport-https
sudo apt install nginx mariadb-server apachetop php-fpm php-mysql php-json php-xml php-gd php-mbstring php-zip php-intl php-imap opendkim opendkim-tools apache2-utils

sudo mysql_secure_installation

# Only for internal mail processing
sudo apt install dovecot-imapd

# Only for Let's Encrypt
sudo apt-get install software-properties-common
sudo add-apt-repository universe
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-nginx
</code>

**Note:** ''apache2-utils'' are for htaccess passwords that are needed for Munin.

**Note:** Try incremental backups with ''mariabackup''.


===== Security =====

==== Security Updates ====

''unattended-upgrades'' has to be activated, see [[https://wiki.debian.org/UnattendedUpgrades|UnattendedUpgrades]]

<code>
sudo dpkg-reconfigure -plow unattended-upgrades
sudo vim /etc/apt/apt.conf.d/50unattended-upgrades
</code>

<file conf  /etc/apt/apt.conf.d/50unattended-upgrades>
Unattended-Upgrade::Mail "info@soscisurvey.de";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "05:00";
</file>


==== Weekly Updates ====

<code>
sudo vim /root/mailinfo.update.sh
</code>

<file bash /root/mailinfo.update.sh>
#!/bin/sh
printf "Subject:[Server] SERVERID Update\n\n%s\n." "`/root/update.sh`" | /usr/sbin/sendmail -t info@soscisurvey.de
</file>

<code>
sudo vim /root/update.sh
</code>

<file bash /root/update.sh>
#!/bin/bash
sudo apt-get update
sudo apt-get --quiet --assume-yes --show-upgraded dist-upgrade
sudo apt-get autoremove
</file>

<code>
sudo vim /root/kernel-clean.sh
</code>

<file bash /root/kernel-clean.sh>
#!/bin/bash
printf "\n\nCurrent Kernel\n"
uname -a

printf "\n\nto delete\n"
dpkg -l 'linux-[ihs]*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\([-0-9]*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | tee kernel-clean.list

printf "\n"
read -p "Press enter to continue"

cat kernel-clean.list | xargs sudo apt-get -y purge
rm kernel-clean.list
</file>

<code>
sudo vim /root/mailinfo.reboot.sh
</code>

<file bash /root/mailinfo.reboot.sh>
#!/bin/sh
printf "Subject:[Server] SERVERID Reboot\n\nServer rebooting\n." | /usr/sbin/sendmail -t info@soscisurvey.de
</file>

<code>
sudo vim /root/postfix.sh
</code>

<file bash /root/postfix.sh>
#!/bin/bash
# Update the postfix databases and reload
postmap /etc/postfix/virtual
postmap /etc/postfix/senders
postalias /etc/aliases
postfix reload
</file>

<code bash>
sudo chmod u+x /root/kernel-clean.sh
sudo chmod u+x /root/update.sh
sudo chmod u+x /root/mailinfo.update.sh
sudo chmod u+x /root/mailinfo.reboot.sh
sudo chmod u+x /root/postfix.sh
</code>


==== Backup-Scripts (extern) ====

See [[https://www.thomas-krenn.com/de/wiki/Backup_unter_Linux_mit_duply|Backup under Linux with duply]].

<code bash>
sudo mkdir /root/backup/
sudo mkdir /root/backup/mysql/
sudo mkdir /mnt/backup
sudo touch /mnt/backup/NOT\ MOUNTED
sudo touch /mnt/backup/SERVERID
sudo vim /etc/fstab
</code>

<code bash>
nas04.partnergate.com:/volume4/sosci /mnt/backup nfs defaults,soft,timeo=30,local_lock=flock,addr=95.130.16.150 0 0
</code>

<code bash>
sudo vim /root/backup/backup-daily.sh
</code>

<file bash /root/backup/backup-daily.sh>
#!/bin/bash
echo "



********** Backup (Daily) **********" >> /var/log/backup-daily.log
date >> /var/log/backup-daily.log
mount /mnt/backup >> /var/log/backup-daily.log
echo "
" >> /var/log/backup-daily.log

/usr/bin/duply database backup_purge --force >> /var/log/backup-daily.log
#/usr/bin/duply sosci-database-A backup_purge --force >> /var/log/backup-daily.log
#/usr/bin/duply sosci-database-B backup_purge --force >> /var/log/backup-daily.log
#/usr/bin/duply server backup_purge --force >> /var/log/backup-daily.log
/usr/bin/duply files backup_purge --force >> /var/log/backup-daily.log

sleep 20
umount /mnt/backup >> /var/log/backup-daily.log
</file>

<code bash>
sudo vim /root/backup/backup-weekly.sh
</code>

<file bash /root/backup/backup-weekly.sh>
#!/bin/bash
echo "


***** Backup (Weekly) *****" >> /var/log/backup-weekly.log
date >> /var/log/backup-weekly.log
mount /mnt/backup >> /var/log/backup-weekly.log
echo "
" >> /var/log/backup-weekly.log

/usr/bin/duply server backup_purge --force >> /var/log/backup-weekly.log
# /usr/bin/duply s2server-etc backup_purge --force >> /var/log/backup-weekly.log
# /usr/bin/duply s2server-root backup_purge --force >> /var/log/backup-weekly.log

# Disk space information sent to standard output

echo "***** Backup Weekly Stats *****
"
/usr/local/nagios/libexec/check_disk -w 15% -c 8% -p /mnt/backup
echo ""
du /mnt/backup --max-depth=2 -h

sleep 20

umount /mnt/backup >> /var/log/backup-weekly.log
</file>

<code bash>
sudo chmod u+x /root/backup/backup-daily.sh
sudo chmod u+x /root/backup/backup-weekly.sh
sudo duply server create
sudo duply database create
sudo duply files create
</code>

<code bash>
cd ~
sudo su
gpg --gen-key

gpg --list-secret-keys
gpg --output private.pem --armor --export-secret-key [KEYID]
gpg --output public.pgp --armor --export [KEYID]
chown sosci:sosci private.pem

# Sichern

rm private.pem
rm public.pgp
</code>

**Important:** Transfer key to RAM disk and save it in KeePass ([[https://sourceforge.net/projects/imdisk-toolkit/|ImDisk]]).

Note key data in KeePass and download ''private.pem'' and save it in KeePass.

<code bash>
cd ~/.duply/server
mv conf conf.dist
vim conf
vim exclude
</code>

<file bash /root/.duply/server/conf>
GPG_KEY='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_PW='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_OPTS="--pinentry-mode loopback"
TARGET='file:///mnt/backup/SERVERID/server'
TARGET_USER=''
TARGET_PASS=''
SOURCE='/'
MAX_AGE=3M
MAX_FULLBKP_AGE=2M
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
</file>

<file bash /root/.duply/server/exclude>
/initrd.img
/lib64
/vmlinuz
/dev/
/proc/
/sys/
/tmp/
/run/
/mnt/
/media/
/lost+found/

/home/sosci/backup/
/usr/share/doc/
/usr/src/
/var/backup/
/var/cache/
/var/www/
/var/tmp/
/var/lib/mysql/
/var/lib/lxcfs/cgroup/
/root/.cache/
/root/backup/mysql/
</file>

<code bash>
cd ~/.duply/database
mv conf conf.dist
vim conf
vim pre
vim post
chmod u+x pre
chmod u+x post
</code>

<file bash /root/.duply/database/conf>
GPG_KEY='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_PW='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_OPTS="--pinentry-mode loopback"
TARGET='file:///mnt/backup/SERVERID/database'
SOURCE='/root/backup/mysql'
MAX_AGE=1M
MAX_FULL_BACKUPS=31
MAX_FULLBKP_AGE=23h
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
</file>

<file bash /root/.duply/database/pre>
#!/bin/sh
datetime=`date +%Y-%m-%d_%H-%M-%S`
mysqldump -ubackup -p[PASSWORD_BACKUP] --databases s2survey --ignore-table sosci.sosci_receivers --ignore-table sosci.sosci_contacts --ignore-table sosci.sosci_users --quick --single-transaction | gzip -9 > /root/backup/mysql/sosci-survey-$datetime.sql.gz
mysqldump -ubackup -p[PASSWORD_BACKUP] --quick --single-transaction s2survey sosci_receivers | gzip -9 > /root/backup/mysql/sosci-receivers-$datetime.sql.gz
mysqldump -ubackup -p[PASSWORD_BACKUP] --quick --single-transaction s2survey sosci_contacts | gzip -9 > /root/backup/mysql/sosci-contacts-$datetime.sql.gz
mysqldump -ubackup -p[PASSWORD_BACKUP] --quick --single-transaction s2survey sosci_users | gzip -9 > /root/backup/mysql/sosci-users-$datetime.sql.gz

</file>

At ''www.soscisurvey.en'' also added....

<file bash /root/.duply/database/pre>
mysqldump -ubackup -p[PASSWORD_BACKUP] --databases rtr_mobile --quick --single-transaction | gzip -9 > /root/backup/mysql/rtr-$datetime.sql.gz
mysqldump -ubackup -p[PASSWORD_BACKUP] --databases distribution --quick --single-transaction | gzip -9 > /root/backup/mysql/customers-$datetime.sql.gz
</file>

<file bash /root/.duply/database/post>
#!/bin/sh
rm /root/backup/mysql/*.sql.gz
</file>

<code bash>
cd ~/.duply/files
mv conf conf.dist
vim conf
vim exclude
</code>

Backup can be done either to the local drive (mounted backup storage) or vis SFTP. The module ''python-paramiko'' (via ''apt'')) may still be required for this.

<file bash /root/.duply/files/conf>
GPG_KEY='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_PW='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_OPTS="--pinentry-mode loopback"
TARGET='file:///mnt/backup/SERVERID/files'
SOURCE='/var/www/s2survey/html'
MAX_AGE=1M
MAX_FULL_BACKUPS=2
MAX_FULLBKP_AGE=14D
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
</file>

<file bash /root/.duply/files/conf>
GPG_KEY='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
GPG_PW=''
TARGET='sftp://s2backup@207.180.212.193/sosci/files'
TARGET_USER='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
TARGET_PASS='XXXXXXXXXXXXXXXXXXXXXXXXXXX'
SOURCE='/var/www/soscisurvey/html'
MAX_AGE=13D
MAX_FULL_BACKUPS=1
MAX_FULLBKP_AGE=1M
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
</file>

<file bash /root/.duply/files/exclude>
/var/www/s2survey/html/view/
/var/www/s2survey/html/system/cache/
/var/www/s2survey/html/system/temp/
/var/www/s2survey/html/system/session/
/var/www/s2survey/html/system/lock/

/var/www/s2survey/html/files/protected/000000/
/var/www/s2survey/html/CUSTOM_PROJECT/
</file>



==== Backup (interal Backu) ====

<code bash>
sudo mkdir /root/backup/
sudo mkdir /root/backup/mysql/
sudo vim /root/backup/backup-daily.sh
</code>

<file bash /root/backup/backup-daily.sh>
#!/bin/sh
find /root/backup/mysql -name "*.gz" -type f -mtime +30 -exec rm -f {} \;

datetime=`date +%Y-%m-%d_%H-%M-%S`
mysqldump -ubackup -pXXXXXXXXXXXXXXXXXXXX --databases s2survey --ignore-table s2survey.s2_receivers --quick --single-transaction | gzip -9 > /root/backup/mysql/s2survey-$datetime.sql.gz
mysqldump -ubackup -pXXXXXXXXXXXXXXXXXXXX --quick --single-transaction s2survey s2_receivers | gzip -9 > /root/backup/mysql/s2survey-receivers-$datetime.sql.gz
</file>


==== Adjust NTP-Server ====

s. [[https://help.ubuntu.com/lts/serverguide/NTP.html|Time Synchronization]]

<code>
sudo vim /etc/systemd/timesyncd.conf
NTP=ptbtime1.ptb.de
FallbackNTP=ntp1.lrz.de
</code>

==== fail2ban ====

Accept the other survey server as trusted.

<code bash>
sudo vim /etc/fail2ban/jail.local
</code>

<file conf /etc/fail2ban/jail.local>
[DEFAULT]
ignoreip = 127.0.0.1/8 95.130.22.98
findtime = 600
bantime  = 600
maxretry = 3
destemail = info@soscisurvey.de
chain = INPUT

[serial-fail]
# SoSci Surveys locks IPs on its own, this is only the upper limit
enabled  = true
port     = http,https
filter   = serial-fail
action   = iptables-allports
           mail-whois[name=Serial Fail, dest=info@soscisurvey.de]
logpath  = /var/www/s2survey/html/system/logfiles/token-fail.log
maxretry = 50
bantime  = 7200

[nginx-forbidden]
# Requests to PHP files that must not be accessed
enabled  = true
filter   = nginx-forbidden
# action   = iptables-allports
action = sendmail-whois-ipmatches[name=Serial Fail, dest=info@soscisurvey.de]
logpath  = /var/www/s2survey/log/n-forbidden.log
maxretry = 2
bantime  = 30


[sshd]
enabled = true
mode    = aggressive

[php-url-fopen]
enabled = true

# Only if installed
[owncloud]
enabled = true
</file>

<code bash>
sudo vim /etc/fail2ban/filter.d/serial-fail.conf
</code>

<file conf /etc/fail2ban/filter.d/serial-fail.conf>
[Definition]
failregex = <HOST>\s+(invalid|serial|deliveryToken)
ignoreregex =
</file>

<code bash>
sudo vim /etc/fail2ban/filter.d/nginx-forbidden.conf
</code>

<file conf /etc/fail2ban/filter.d/nginx-forbidden.conf>
# Fail2Ban filter to match web requests for "access denied" URLs
#

[INCLUDES]

# Load regexes for filtering
# before = botsearch-common.conf

[Definition]

failregex = ^<HOST> \- \- .*? \"(GET|POST|HEAD) .*?\" 403 .+$

ignoreregex =

datepattern = {^LN-BEG}%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?
              ^[^\[]*\[({DATE})
              {^LN-BEG}
</file>

<code bash>
sudo vim /etc/fail2ban/filter.d/owncloud.conf
</code>

<file>
[Definition]
failregex={"reqId":".*","remoteAddr":".*","app":"core","message":"Login failed: '.*' \(Remote IP: '<HOST>'\)","level":2,"time":".*"}

ignoreregex =
</file>
==== rkhunter ====

''xinet '' os allowed for Nagios.

<code>
sudo vim /etc/rkhunter.conf

PKGMGR=DPKG
XINETD_ALLOWED_SVC=/etc/xinetd.d/nrpe

sudo rkhunter --propupd

sudo vim /etc/default/rkhunter
CRON_DAILY_RUN="true"
CRON_DB_UPDATE="true"
REPORT_EMAIL="info@soscisurvey.de"
APT_AUTOGEN="true"
</code>

<code bash>
mkdir /var/log/chkrootkit/
sudo vim /etc/chkrootkit.conf
RUN_DAILY="true"
</code>


===== Settings =====

    * Daily Backup (MySQL und Dateien) via Cronjob
      * [[http://www.howtoforge.de/anleitung/verschlusselte-ftp-backups-mit-duplicity-und-duply-erstellen-debian-squeeze/|Verschlüsselte FTP Backups mit duplicity und duply erstellen]]
    *  Enable retrieval of server status [[http://httpd.apache.org/docs/2.2/mod/mod_status.html|Apache Module mod_status]]



===== Firewall =====

As firewall ''iptables'' is used, so that no ports are accidentally open after except. It also allows you to block some attacks.

<code>
sudo mkdir /etc/iptables
# sudo bash -c "iptables-save > /etc/iptables/rules.v4"
# sudo bash -c "ip6tables-save > /etc/iptables/rules.v6"
# Gleiches Regelset für IPv4 un IPv6
cd /etc/iptables
sudo vim rules.base
sudo ln -s rules.base rules.v4
sudo ln -s rules.base rules.v6
</code>

  * The rules are stored in ''/etc/iptables/rules.base'', the files ''rules.v4'' and ''rules.v6'' only refer to this file to avoid duplication and misconfiguration.
  * For testing (IPv4 and IPv4 separated!) very well suitable: [[http://www.ipv6scanner.com/]]

<file conf rules.base>
## Custom rules
## Based on https://gist.github.com/jirutka/3742890
*filter
# 
# Base policy
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# 
# Drop non-conforming packets, such as malformed headers, etc.
-A INPUT -m conntrack --ctstate INVALID -j DROP
#
# Antispoofing
# Block remote packets claiming to be from a loopback address.
-4 -A INPUT -s 127.0.0.0/8 ! -i lo -j DROP
-6 -A INPUT -s ::1/128 ! -i lo -j DROP
-6 -A INPUT -i $WAN_IF -s FC00::/7 -j DROP
-6 -A FORWARD -s ::1/128 -j DROP
-6 -A FORWARD -i $WAN_IF -s FC00::/7 -j DROP
#
# Don't attempt to firewall internal traffic on the loopback device.   
-A INPUT -i lo -j ACCEPT
-A INPUT -s localhost -j ACCEPT
#
# Whitelist Monitoring Server
-4 -A INPUT -s 95.130.16.86 -j ACCEPT
-4 -A INPUT -s 95.130.16.67 -j ACCEPT
#
# Continue connections that are already established or related to an established
# connection.
-4 -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-6 -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
#
# Drop all packets that are going to broadcast, multicast or anycast address.
-4 -A INPUT -m addrtype --dst-type BROADCAST -j DROP
-4 -A INPUT -m addrtype --dst-type MULTICAST -j DROP
-4 -A INPUT -m addrtype --dst-type ANYCAST -j DROP
-4 -A INPUT -d 224.0.0.0/4 -j DROP
#
# Chain for preventing ping flooding - up to 6 pings per second from a single
# source, again with log limiting. Also prevents us from ICMP REPLY flooding
# some victim when replying to ICMP ECHO from a spoofed source.
-N ICMPFLOOD
-A ICMPFLOOD -m recent --set --name ICMP --rsource
-A ICMPFLOOD -m recent --update --seconds 1 --hitcount 6 --name ICMP --rsource --rttl -m limit --limit 1/sec --limit-burst 1 -j LOG --log-prefix "iptables[ICMP-flood]: "
-A ICMPFLOOD -m recent --update --seconds 1 --hitcount 6 --name ICMP --rsource --rttl -j DROP
-A ICMPFLOOD -j ACCEPT
#
# Chain for preventing SSH brute-force attacks.
# Permits 10 new connections within 5 minutes from a single host then drops
# incomming connections from that host. Beyond a burst of 100 connections we
# log at up 1 attempt per second to prevent filling of logs.
-N SSHBRUTE
-A SSHBRUTE -m recent --name SSH --set
-A SSHBRUTE -m recent --name SSH --update --seconds 300 --hitcount 10 -m limit --limit 1/second --limit-burst 100 -j LOG --log-prefix "iptables[SSH-brute]: "
-A SSHBRUTE -m recent --name SSH --update --seconds 300 --hitcount 10 -j DROP
#
# Allow SSH from certain IPs
# Allow from www.onlineforschung.org
-4 -A SSHBRUTE -s 207.180.212.193 -j ACCEPT
# Allow from s2survey.net
-4 -A SSHBRUTE -s 95.130.22.100 -j ACCEPT
# Allow from MWN/LMU
-4 -A SSHBRUTE -s 138.246.2.0/24 -j ACCEPT
# Allow from LMU/VPN LRZ_VPN_public_LMU_IPv6
-6 -A SSHBRUTE -s 2001:4ca0:4fff::/48 -j ACCEPT
# Optionally allow from m-net
-4 -A SSHBRUTE -s 62.216.206.12/24 -j ACCEPT
-4 -A SSHBRUTE -s 62.216.202.189/24 -j ACCEPT
-A SSHBRUTE -j DROP
#
# NUL Packages (Portscanner) blockieren
-A INPUT -p tcp --tcp-flags ALL NONE -j DROP
# SYN flood blockieren
-A INPUT -p tcp ! --syn -m state --state NEW -j DROP
# XMAS blockieren
-A INPUT -p tcp --tcp-flags ALL ALL -j DROP
#
#
## SMTP only if receiving emails
# -A INPUT -p tcp -m tcp --dport 25 -j ACCEPT
#
## HTTP/S
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
#
## Monit (port 2812)
-A INPUT -p tcp -m tcp --dport 2812 -m state --state NEW -m limit --limit 5/min --limit-burst 20 -j ACCEPT
#
#
# Test Ports
#-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
#-A INPUT -p tcp -m tcp --dport 8081 -j ACCEPT									   
#
# Ping (outgoing)
-A OUTPUT -p icmp -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
#
###############################################################################
# 3. GENERAL RULES                                                            #
#                                                                             #
# This section contains general rules that should be suitable for most hosts. #
###############################################################################
#
# Accept worldwide access to SSH and use SSHBRUTE chain for preventing and filter IPs
-4 -A INPUT -p tcp --dport 22 --syn -m state --state NEW -j SSHBRUTE
-6 -A INPUT -p tcp --dport 22 --syn -m conntrack --ctstate NEW -j SSHBRUTE
#
# Permit useful IMCP packet types for IPv4
# Note: RFC 792 states that all hosts MUST respond to ICMP ECHO requests.
# Blocking these can make diagnosing of even simple faults much more tricky.
# Real security lies in locking down and hardening all services, not by hiding.
-4 -A INPUT -p icmp --icmp-type 0  -m conntrack --ctstate NEW -j ACCEPT
-4 -A INPUT -p icmp --icmp-type 3  -m conntrack --ctstate NEW -j ACCEPT
-4 -A INPUT -p icmp --icmp-type 11 -m conntrack --ctstate NEW -j ACCEPT
#
# Permit needed ICMP packet types for IPv6 per RFC 4890.
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 1   -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 2   -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 3   -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 4   -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 133 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 134 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 135 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 136 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 137 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 141 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 142 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 130 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 131 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 132 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 143 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 148 -j ACCEPT
-6 -A INPUT              -p ipv6-icmp --icmpv6-type 149 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 151 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 152 -j ACCEPT
-6 -A INPUT -s fe80::/10 -p ipv6-icmp --icmpv6-type 153 -j ACCEPT
#
# Permit IMCP echo requests (ping) and use ICMPFLOOD chain for preventing ping
# flooding.
-4 -A INPUT -p icmp --icmp-type 8  -m conntrack --ctstate NEW -j ICMPFLOOD
-6 -A INPUT -p ipv6-icmp --icmpv6-type 128 -j ICMPFLOOD
#								   
#
# Do not log packets that are going to ports used by SMB
# (Samba / Windows Sharing).
-A INPUT -p udp -m multiport --dports 135,445 -j DROP
-A INPUT -p udp --dport 137:139 -j DROP
-A INPUT -p udp --sport 137 --dport 1024:65535 -j DROP
-A INPUT -p tcp -m multiport --dports 135,139,445 -j DROP
#
# Do not log packets that are going to port used by UPnP protocol.
-A INPUT -p udp --dport 1900 -j DROP
#
# Do not log late replies from nameservers.
-A INPUT -p udp --sport 53 -j DROP
#
# Good practise is to explicately reject AUTH traffic so that it fails fast.
-A INPUT -p tcp --dport 113 --syn -m conntrack --ctstate NEW -j REJECT --reject-with tcp-reset
#
# Prevent DOS by filling log files.
-A INPUT -m limit --limit 1/second --limit-burst 100 -j LOG --log-prefix "iptables[DOS]: "
#
# Final action
-A INPUT -j DROP
#
COMMIT
</file>

  * Port 25 (incoming) is only enabled on ''mail''.

Load the configuration on reboot (from Ubuntu 18), cf. [[https://nucco.org/2018/05/ubuntu-18-04-chronicles-applying-firewall-rules-on-startup-pre-network.html|sudo systemctl enable iptables-load.service]]

<code bash>
sudo vim /etc/iptables/restore.sh
sudo vim /etc/systemd/system/iptables-load.service
sudo chmod u+x /etc/iptables/restore.sh
</code>

<file bash /etc/iptables/restore.sh>
#!/bin/bash
/sbin/iptables-restore < /etc/iptables/rules.v4
/sbin/ip6tables-restore < /etc/iptables/rules.v6
</file>

<file bash /etc/systemd/system/iptables-load.service>
[Unit]
Description = Load firewall rules on startup

[Service]
Type=oneshot
ExecStart=/etc/iptables/restore.sh

[Install]
WantedBy=network-pre.target
</file>

Activate the service and start it right away to test it (''--now'').

<code bash>
sudo systemctl enable iptables-load.service --now
</code>


==== Protection against DOS attacks (optional) ====

  * [[https://javapipe.com/blog/iptables-ddos-protection/|Limit connections against DoS attacks]]


===== SFTP ======

Set up FTP user that can only access SoSci Survey.

<code bash>
sudo groupadd www-ftp
sudo useradd -g www-ftp -m -d /home/www-ftp -s /sbin/nologin www-ftp
sudo mkdir /home/www-ftp/.ssh
sudo vim /home/www-ftp/.ssh/authorized_keys
# Schlüssel aus PuttyGen einsetzen
sudo chown -R www-ftp:www-ftp /home/www-ftp/.ssh
sudo chmod 700 /home/www-ftp/.ssh/
sudo chmod 600 /home/www-ftp/.ssh/authorized_keys
</code>

Create directories for SoSci Survey

<code bash>
sudo mkdir /var/www/s2survey
sudo mkdir /var/www/s2survey/html
sudo mkdir /var/www/s2survey/log/
sudo chown root:root /var/www/s2survey
sudo chown -R www-data:www-data /var/www/s2survey/log
sudo chown -R www-ftp:www-data /var/www/s2survey/html
sudo chmod 770 /var/www/s2survey/html
</code>

<file conf /etc/passwd>
www-ftp:x:1003:1003::/html:/usr/sbin/nologin
</file>

<file conf /etc/ssh/sshd_config>
#Subsystem       sftp    /usr/libexec/openssh/sftp-server
Subsystem       sftp    internal-sftp

# Now at the end (!) of the file
Match User www-ftp
    AuthorizedKeysFile /home/www-ftp/.ssh/authorized_keys
    ChrootDirectory /var/www/s2survey
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

Match all
</file>

<code bash>
sudo systemctl reload sshd
</code>

If there are problems, run ''sshd'' in debug mode:

<code bash>
sudo /usr/sbin/sshd -d -p 3321
</code>

===== Webserver =====

==== PHP-FPM ====

The default pool ''www.conf'' must be deleted or renamed, e.g. ''mv www.conf www.conf.dpkg-dist''.

The setting ''pm.max_children = 32'' is designed for 6 GB RAM and elevations with heavy rushes.

<code bash>
sudo mv /etc/php/7.4/fpm/pool.d/www.conf /etc/php/7.4/fpm/pool.d/www.conf.dist
sudo vim /etc/php/7.4/fpm/pool.d/primary.conf
sudo vim /etc/php/7.4/fpm/pool.d/secondary.conf

sudo mv /etc/php/8.0/fpm/pool.d/www.conf /etc/php/8.0/fpm/pool.d/www.conf.dist
sudo vim /etc/php/8.0/fpm/pool.d/primary.conf
sudo vim /etc/php/8.0/fpm/pool.d/secondary.conf
</code>

<file conf /etc/php/7.4/fpm/pool.d/primary.conf>
[primary]
user = www-data
group = www-data
listen = /run/php/php-fpm.sock
listen.backlog = 1023
listen.owner = www-data
listen.group = www-data
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 32
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 2500
pm.status_path = /status.primary
ping.path = /ping.primary
env[PATH] = /usr/local/bin:/usr/bin:/bin
# evtl. für SimpleSAML
env[SIMPLESAMLPHP_CONFIG_DIR] = /var/www/shibboleth/config
</file>

<file conf /etc/php/7.4/fpm/pool.d/secondary.conf>
[secondary]
user = www-data
group = www-data
listen = 127.0.0.1:9002
listen.backlog = 511
listen.owner = www-data
listen.group = www-data
listen.allowed_clients = 127.0.0.1
pm = ondemand
pm.max_children = 10
pm.process_idle_timeout = 10s;
pm.max_requests = 500
pm.status_path = /status.secondary
ping.path = /ping.secondary
env[PATH] = /usr/local/bin:/usr/bin:/bin
</file>

And then a few changes in the php.ini (former ''/etc/php/7.4/fpm/conf.d/10-sosci.ini'')

<file conf /etc/php/8.0/fpm/conf.d/10-sosci.ini>
display_errors = On
display_startup_errors = On
post_max_size = 256M
upload_max_filesize = 256M
max_file_uploads = 50
mail.add_x_header = Off
</file>


===== nginx =====

Copy certificates and keys to ''/var/www/ssl/''.

<code bash>
sudo vim /etc/nginx/nginx.conf

# Delete
ssl_prefer_server_ciphers
ssl_protocols
</code>

The actual configuration is then placed in ''/etc/nginx/conf.d''.

<file conf /etc/nginx/conf.d/ssl.conf>
# SSL Settings

ssl_protocols TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE, and TLSv1
ssl_prefer_server_ciphers on;
ssl_ciphers EECDH+AESGCM:EDH+AESGCM;
ssl_ecdh_curve secp384r1;
ssl_dhparam /var/www/ssl/dhparam.pem;
## Use a SSL/TLS cache for SSL session resume.
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_stapling on;
ssl_stapling_verify on;
# HTTP-Strict Transport Security (HSTS)
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";

# Information disclosure
server_tokens off;
</file>

<file conf /etc/nginx/conf.d/logs.conf>
# Logging Settings

log_format noip '0.0.0.0 - $remote_user [$time_local]  '
    '"$request" $status $body_bytes_sent '
    '"$http_referer" "$http_user_agent"';

log_format noagent '0.0.0.0 - $remote_user [$time_local]  '
    '"$request" $status $body_bytes_sent '
    '"$http_referer" "na"';
</file>

<file conf /etc/nginx/conf.d/gzip.conf>
gzip_disable "msie6";
gzip_vary on;
</file>

<file conf /etc/nginx/fastcgi_params>
fastcgi_connect_timeout 65;
fastcgi_send_timeout    180;
fastcgi_read_timeout    900;
</file>

<file conf /etc/logrotate.d/nginx>
/var/log/nginx/*.log
/var/www/s2survey/log/*.log {
        weekly
        missingok
        rotate 28
        dateext
        compress
        delaycompress
        notifempty
        create 0640 www-data adm
        sharedscripts
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi \
        endscript
        postrotate
                # [ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
                invoke-rc.d nginx rotate >/dev/null 2>&1
        endscript
}
</file>

<file conf /etc/nginx/mime.types>
+
application/wasm                      wasm;
</file>

<code bash>
systemctl stop php7.4-fpm
systemctl start php7.4-fpm
</code>


==== MySQL ====

<code bash>
sudo vim /etc/mysql/conf.d/s2survey.cnf
sudo systemctl restart mysql
</code>

<file conf /etc/mysql/conf.d/s2survey.cnf>
[mysqld]
key_buffer_size         = 64M
max_allowed_packet      = 32M
max_connections         = 512

slow_query_log          = 1
slow_query_log_file     = /var/log/mysql/mysql-slow.log
long_query_time         = 5

# InnoDb configuration
innodb_file_per_table = 1
innodb_buffer_pool_size = 2G # 50% of System Memory was better
innodb_log_file_size=128M
innodb_log_buffer_size = 512M
innodb_thread_concurrency = 8
innodb_lock_wait_timeout=20
# innodb_additional_mem_pool_size = 20M
# innodb_data_file_path = ibdata1:50M:autoextend

# pid-file      = /var/run/mysqld/mysqld.pid

skip-log-bin
# binlog_expire_log_seconds = 172800
</file>

<code bash>
systemctl stop mysql
systemctl start mysql
sudo su
mysql
</code>

=== Data Dictionary MySQL ===

The [[https://www.digitalocean.com/community/tutorials/how-to-move-a-mysql-data-directory-to-a-new-location-on-ubuntu-16-04|Data Dictionary of MySQL]] might have to be moved.

<code bash>
sudo systemctl stop mysql
sudo mv /var/lib/mysql /ENCRYPTED/mysql
sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

datadir=/ENCRYPTED/mysql

sudo systemctl start mysql
</code>

=== Encryption ===

Then we still use the [[https://mariadb.com/kb/en/data-at-rest-encryption-overview/|Data-at-Rest-Encryption of MariaDB]].

<code bash>
sudo mkdir /etc/mysql/encryption
sudo touch /etc/mysql/encryption/keyfile
sudo chown mysql:mysql /etc/mysql/encryption/keyfile
sudo chmod 600 /etc/mysql/encryption/keyfile
sudo sh -c 'echo -n "1;" >> /etc/mysql/encryption/keyfile'
sudo sh -c 'openssl rand -hex 32 >> /etc/mysql/encryption/keyfile'

sudo less /etc/mysql/encryption/keyfile

sudo touch /etc/mysql/conf.d/panel.cnf
sudo chown mysql:mysql /etc/mysql/conf.d/panel.cnf
sudo chmod 600 /etc/mysql/conf.d/panel.cnf

sudo vim /etc/mysql/conf.d/panel.cnf
</code>

<code conf>
[mariadb]
plugin_load_add = file_key_management
loose_file_key_management_filename = /etc/mysql/encryption/keyfile
loose_file_key_management_encryption_algorithm = AES_CTR

# InnoDB/XtraDB Encryption
innodb_encrypt_tables = ON
innodb_encrypt_temporary_tables = ON
innodb_encrypt_log = ON
innodb_encryption_threads = 4
innodb_encryption_rotate_key_age = 1

# InnoDb configuration
innodb_file_per_table = 1
innodb_buffer_pool_size = 2G
innodb_log_file_size=128M
innodb_log_buffer_size = 512M
innodb_thread_concurrency = 8
innodb_lock_wait_timeout=20

[mariabackup]
user=mariabackup
password=###PWD###
</code>

Backup the contents of the keyfile to KeePass.

<code bash>
sudo systemctl restart mariadb
</code>

After creating a table, you can then look in the table ''`information_schema`.`INNODB_TABLESPACES_ENCRYPTION`'' to see if encryption is active. There ''ENCRYPTION_SCHEME'' and ''CURRENT_KEY_ID'' must have the value 1.

=== MySQL-User ===

Create a new root user with password, see [[https://tableplus.io/blog/2018/10/how-to-create-a-superuser-in-mysql.html|https://tableplus.io/blog/2018/10/how-to-create-a-superuser-in-mysql.html]]

<code mysql>
CREATE USER 'master'@'localhost' IDENTIFIED BY 'XXXXXXXXXXXXXX';
GRANT ALL PRIVILEGES ON *.* TO 'master'@'localhost' WITH GRANT OPTION;
SHOW GRANTS FOR master@localhost;
FLUSH PRIVILEGES;

CREATE DATABASE s2survey;
CREATE USER 's2survey'@'localhost' IDENTIFIED BY 'XXXXXXXXXXXXXX';
CREATE USER 'backup'@'localhost' IDENTIFIED BY 'XXXXXXXXXXXXXX';
GRANT ALL PRIVILEGES ON s2survey.* TO 's2survey'@'localhost';
GRANT USAGE ON *.* TO 'backup'@'localhost';
GRANT SELECT, LOCK TABLES ON `mysql`.* TO 'backup'@'localhost';
GRANT SELECT, LOCK TABLES, SHOW VIEW, EVENT, TRIGGER ON `myschema`.* TO 'backup'@'localhost';
FLUSH PRIVILEGES;

QUIT;
</code>


==== Let's Encrypt ====

And maybe the configuration must be copied to the ''letsencrypt'' as well. To do this, the directory on one server must first be transferred to the user ''sosci'':

<code bash>
sudo chown -R sosci:sosci /etc/letsencrypt/
</code>

Then must be copied on the other server with symlinks:

<code bash>
sudo rsync -avz -e ssh sosci@s2survey.net:/etc/letsencrypt /etc/
</code>

And then the file owners restored on the one server

<code bash>
sudo chown -R root:root /etc/letsencrypt/
</code>

To test it...

<code bash>
sudo /usr/bin/certbot renew --post-hook "service nginx reload"
</code>


==== Websites ====

<code bash>
sudo vim /etc/nginx/nginx.conf

# im http-Block ergänzen
include /etc/nginx/sites-enabled/*;

sudo mkdir /etc/nginx/sites-available
sudo mkdir /etc/nginx/sites-enabled
</code>

First create a Self-Signed Certificate

<code bash>
sudo su
cd /var/www/ssl
openssl req -x509 -nodes -newkey rsa:2048 -keyout selfsigned.key -out selfsigned.crt -days 365
</code>


<file conf /etc/nginx/sites-available/01.s2survey.conf>
# PHP-FPM processes to be used by SoSci Survey
upstream sosci {
    server unix:/run/php/php-fpm.sock max_fails=5 fail_timeout=30s;
    server 127.0.0.1:9002 backup;
}

# Limitation for requets per second
# Expect no more than 5 PHP requests (not images) per second = 300 per minute = 30 people doing 10 pages per minute
limit_req_zone $binary_remote_addr zone=php:10m rate=5r/s;
# (Much) less login attempts - for the SoSci Panel at the moment
limit_req_zone $binary_remote_addr zone=login:10m rate=15r/m;

# Main domain(s)
server {
        listen [::]:443 default ipv6only=on;
        listen 443 ssl http2;

        server_name s2survey.net s2survey.de;

        # The server certificate must appear before the chained certificates in the combined file
        # cat www.example.com.crt bundle.crt > www.example.com.chained.crt
        # ssl_certificate_key /etc/letsencrypt/live/s2survey.net/privkey.pem; # fallback
        # ssl_certificate /etc/letsencrypt/live/s2survey.net/fullchain.pem; # fallback
        ssl_certificate_key /var/www/ssl/selfsigned.key;
        ssl_certificate /var/www/ssl/selfsigned.crt;
        # ssl_certificate_key /var/www/ssl/s2survey.2017.key;
        # ssl_certificate /var/www/ssl/s2survey.2018.cert;

        # Content
        root /var/www/s2survey/html;
        index index.php index.html index.htm;
        error_page 404 = /spellcheck.php;
        client_max_body_size    80M;

        access_log /var/www/s2survey/log/n-access.log noagent;
        error_log /var/www/s2survey/log/n-error.log;

        # Forbidden directories
        location ~ ^/(inc|lib|system)/ {
                deny all;
                return 403;
        }
        location ~ ^/files/(protected|upload|share)/ {
                deny all;
                return 403;
        }
        location ~ ^/download/(release|system|working)/ {
                deny all;
                return 403;
        }
        
        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        location ~ /\.ht {
                deny all;
                return 403;
        }

        # Munin
        location ^~ /munin/static/ {
                alias /etc/munin/static/;
                expires modified +1w;
        }

        location ^~ /munin/ {
                alias /var/cache/munin/www/;
                expires modified +310s;
                try_files $uri $uri/ =404;
                auth_basic "Munin";
                auth_basic_user_file /etc/munin/munin-htpasswd;
        }
        
        # Monit via SSL
        location ^~ /monit/ {
                rewrite ^/monit/(.*) /$1 break;
                proxy_ignore_client_abort on;
                proxy_pass   http://127.0.0.1:2812;
                proxy_redirect  http://127.0.0.1:2812 /monit;
                proxy_cookie_path / /monit/;
        }
        
        # PHP FPM status
        location ~ ^/(status\.primary|ping\.primary)$ {
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_pass unix:/run/php/php-fpm.sock;
                log_not_found off;
                allow 95.130.22.98;
                allow 2a02:2940:0:c007::98;
                allow 127.0.0.1;
                allow ::1;
                deny all;
        }
        location ~ ^/(status\.secondary|ping\.secondary)$ {
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_pass 127.0.0.1:9002;
                log_not_found off;
                allow 95.130.22.98;
                allow 2a02:2940:0:c007::98;
                allow 127.0.0.1;
                allow ::1;
                deny all;
        }
        location ~ ^/nginx_status$ {
                stub_status;
                log_not_found off;
                allow 95.130.22.98;
                allow 2a02:2940:0:c007::98;
                allow 127.0.0.1;
                allow ::1;
                deny all;
        }
        
        # Different PHP settings for the sosci administration
        location ~ ^/admin/(.+\.php)$ {
                limit_req zone=php burst=50;
                try_files $uri =404;
                # No secondary pool for that
                fastcgi_pass unix:/run/php/php-fpm.sock;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                include fastcgi_params;
                # Special
                fastcgi_param PHP_VALUE "
                        max_execution_time = 20
                        memory_limit = 256M
                        max_file_uploads = 50
                        post_max_size = 80M
                ";
        }

        # pass the survey's PHP scripts to FastCGI server listening on 127.0.0.1:9000
        # location ~ (/index.php|^/(help|shop)/(.+\.php)|^/images/wedge.php|^/spellcheck.php|^/download/(activate|download|report)\.php|^/homepage/admin/(.+\.php)|^/tools/view-chars.php)$ {
        location ~ (/index.php|^/images/wedge.php|^/spellcheck.php)$ {
                limit_req zone=php burst=50;
                try_files $uri =404;

                # First, you need to capture fastcgi errors
                fastcgi_intercept_errors      on;
                # Specifies in which cases a request should be passed to the next server
                fastcgi_next_upstream         error timeout http_500 http_503;
                # Limits the time during which a request can be passed
                # to the next server
                fastcgi_next_upstream_timeout 10s;
                # Limits the number of possible tries for passing
                # a request to the next server
                fastcgi_next_upstream_tries   2;
                # Upstream to pass request to
                fastcgi_pass sosci;

                fastcgi_index index.php;
                fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                include fastcgi_params;
                # Limitations for survey projects
                fastcgi_param PHP_VALUE "
                        max_execution_time = 5
                        memory_limit = 64M
                        max_file_uploads = 5
                        post_max_size = 16M
                ";
        }

        # Deny all PHP files that are not index.php or /admin/       
        location ~ \.php$ {
                access_log /var/www/s2survey/log/n-forbidden.log;
                if (-f $request_filename) {
                     return 403;
                }
                if (!-f $request_filename) {
                     return 404;
                }
        }


        # Docs
        location /doc/ {
                alias /usr/share/doc/;
                autoindex on;
                allow 127.0.0.1;
                allow 95.130.22.100;
                deny all;
        }

        # Cache management
        location ~* \.(?:ico|css|js|gif|jpg|jpeg|png|flv|pdf|swf)$ {
                expires 1d;
                add_header Vary Accept-Encoding;
                add_header X-Content-Type-Options nosniff;
                # access_log off;
        }

        # Anything else
        location ~ / {
                # First attempt to serve request as file, then
                # as directory, then fall back to index.html
                try_files $uri $uri/ =404; #$uri/ /index.php /index.html =404;
        }

}
</file>

<file conf /etc/nginx/sites-available/02.customers.conf>
server {
        listen 95.130.22.101:80;
        listen [2a02:2940:0:c007::101]:80;
        server_name
                access.lgbti-label.ch
                verbaendeumfrage2019.divv.de;

        # Allow certbot auth
        root     /var/www/s2survey/certbot;
        location ~ ^/.well-known/ {
        }

        # redirect everything else
        location ~ / {
                # add_header X-Debug "$host";
                rewrite        ^ https://$host$request_uri? permanent;
        }

        access_log /var/www/s2survey/log/n-access-custom.log noagent;
        error_log /var/www/s2survey/log/n-error-custom.log;
}

server {
        listen 95.130.22.101:443 ssl http2;
        listen [2a02:2940:0:c007::101]:443 ssl http2;
        server_name access.lgbti-label.ch;

        root /var/www/s2survey/html/_$host;
        index index.php index.html index.htm;
        client_max_body_size    64M;

        access_log /var/www/s2survey/log/n-access-custom.log noagent;
        error_log /var/www/s2survey/log/n-error-custom.log;

        ssl_certificate_key /etc/letsencrypt/live/access.lgbti-label.ch/privkey.pem;
        ssl_certificate /etc/letsencrypt/live/access.lgbti-label.ch/fullchain.pem;

        location ~ /\.ht {
                deny all;
                return 403;
        }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        location ~ \.php$ {
                limit_req zone=php burst=50;
                try_files $uri =404;

                fastcgi_intercept_errors      on;
                fastcgi_next_upstream         error timeout http_500 http_503;
                fastcgi_next_upstream_timeout 10s;
                fastcgi_next_upstream_tries   2;
                fastcgi_pass sosci;

                fastcgi_index index.php;
                fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                include fastcgi_params;
                # Limitations for survey projects
                fastcgi_param PHP_VALUE "
                        max_execution_time = 5
                        memory_limit = 64M
                        max_file_uploads = 5
                        post_max_size = 16M
                ";
        }

        # Cache management
        location ~* \.(?:ico|css|js|gif|jpg|jpeg|png|flv|pdf|swf)$ {
                expires 1d;
                add_header Vary Accept-Encoding;
                add_header X-Content-Type-Options nosniff;
                # access_log off;
        }

        # Anything else
        location ~ / {
                # First attempt to serve request as file, then
                # as directory, then fall back to index.html
                try_files $uri $uri/ =404; #$uri/ /index.php /index.html =404;
        }

}
</file>

<file conf /etc/nginx/sites-available/06.redirect.conf>
# HTTP to HTTPS redirects
server {
        listen 80;
        listen [::]:80;
        server_name     s2survey.net www.s2survey.net s2survey.com www.s2survey.com s2survey.de www.s2survey.de;
        rewrite         ^ https://$server_name$request_uri? permanent;
}

# (Sub-)Domains to be redirected
server {
        listen 443 ssl http2;
        listen [2a02:2940:0:c007::101]:443 ssl http2;

        server_name www.s2survey.net s2survey.com www.s2survey.com;

        # The server certificate must appear before the chained certificates in the combined file
        # cat www.example.com.crt bundle.crt > www.example.com.chained.crt
        ssl_certificate_key /etc/letsencrypt/live/s2survey.net/privkey.pem; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/s2survey.net/fullchain.pem; # managed by Certbot

        rewrite         ^ https://s2survey.net$request_uri? permanent;



}

# (Sub-)Domains to be redirected
server {
        listen 443 ssl http2;
        listen [2a02:2940:0:c007::101]:443 ssl http2;

        server_name www.s2survey.de;

        ssl_certificate_key /etc/letsencrypt/live/s2survey.net/privkey.pem; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/s2survey.net/fullchain.pem; # managed by Certbot

        rewrite         ^ https://s2survey.de$request_uri? permanent;

}
</file>

<file conf /etc/nginx/sites-available/09.nossl.s2survey.conf>
# Allows access without SSL for single projects
server {
        listen 80;
        listen [::]:80;

        root /var/www/soscisurvey/html;
        index index.php index.html index.htm;
        error_page 404 = /spellcheck.php;

        server_name nossl.s2survey.net;

        access_log /var/www/soscisurvey/log/nossl-access.log noip;
        error_log /var/www/soscisurvey/log/nossl-error.log;


        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        location ~ /\.ht {
                deny all;
        }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        location ~ \.php$ {
                limit_req zone=php burst=50;
                try_files $uri =404;

                fastcgi_intercept_errors      on;
                fastcgi_next_upstream         error timeout http_500 http_503;
                fastcgi_next_upstream_timeout 10s;
                fastcgi_next_upstream_tries   2;
                fastcgi_pass sosci;

                fastcgi_index index.php;
                fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                include fastcgi_params;
                # Limitations for survey projects
                fastcgi_param PHP_VALUE "
                        max_execution_time = 5
                        memory_limit = 64M
                        max_file_uploads = 5
                        post_max_size = 16M
                ";
        }
        
        # Allowed directories
        location /images/ {
                try_files $uri =404;
                allow all;
        }
        location /view/ {
                try_files $uri =404;
                allow all;
        }

        # Allowed projects
        location /curtest/ {
                try_files $uri $uri/ =404;
                allow all;
        }

        location / {
                allow 127.0.0.1;
                deny all;
                try_files $uri $uri/ =404;
        }
}
</file>

<file conf /etc/nginx/sites-available/default>
server {
        listen 80;
        listen [::]:80;

        root /var/www/default/html;
        index index.php index.html index.htm;

        # Make site accessible from http://localhost/
        server_name localhost;

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        location ~ /\.ht {
                deny all;
        }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        location ~ \.php$ {
                limit_req zone=php burst=50;
                try_files $uri =404;
                fastcgi_pass sosci;

                fastcgi_index index.php;
                fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                include fastcgi_params;
        }

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to index.html
                try_files $uri $uri/ =404;
                # Uncomment to enable naxsi on this location
                # include /etc/nginx/naxsi.rules
        }

        # Server status
        location = /server-status {
                # copied from http://blog.kovyrin.net/2006/04/29/monitoring-nginx-with-rrdtool/
                stub_status on;
                #extended_status on;
                access_log   off;
                allow 127.0.0.1;
                allow 95.130.22.98;
                deny all;
        }

        # Server status for munin
        location = /nginx_status {
                stub_status on;
                access_log   off;
                allow 127.0.0.1;
                allow 95.130.22.98;
                deny all;
        }
        
        # Also access munin via localhost
        location ^~ /munin/static/ {
                alias /etc/munin/static/;
                expires modified +1w;
        }

        location ^~ /munin/ {
                alias /var/cache/munin/www/;
                expires modified +310s;
                try_files $uri $uri/ =404;
                auth_basic "Munin";
                auth_basic_user_file /etc/munin/munin-htpasswd;
        }

}
</file>

Then still activate the pages

<code bash>
sudo ln -s /etc/nginx/sites-available/01.s2survey.conf /etc/nginx/sites-enabled/
sudo ln -s /etc/nginx/sites-available/02.customers.conf /etc/nginx/sites-enabled/
sudo ln -s /etc/nginx/sites-available/06.redirect.conf /etc/nginx/sites-enabled/
sudo ln -s /etc/nginx/sites-available/09.nossl.s2survey.conf /etc/nginx/sites-enabled/
</code>


===== E-Mail =====

A separate server is responsible for incoming e-mails, configured according to [[https://thomas-leister.de/mailserver-debian-stretch/|thomas-leister.de]].

Unfortunately, the server SSL keys cannot be used directly for TLS, so we will create new ones first.

<code>
sudo openssl req -new -x509 -days 3650 -nodes -out /etc/ssl/certs/postfix.pem -keyout /etc/ssl/private/postfix.key

Country Name (2 letter code) [AU]:                            DE
State or Province Name (full name) [Some-State]:              Bayern
Locality Name (eg, city) []:                                  Muenchen
Organization Name (eg, company) [Internet Widgits Pty Ltd]:   SoSci Survey GmbH
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:               s2survey.net
Email Address []:                                             info@soscisurvey.de
</code>

To make it work with outgoing mail and mail bounce detection, the following changes are required.

**Note:** On www.soscisurvey.de the returns are not processed directly, but instead by ''mail.soscisurvey.com:143''. This is the unencrypted IMAP port.


==== Postfix ====

<file conf /etc/postfix/main.cf>
##### Use Maildir for IMAP/dovecot #####
home_mailbox = Maildir/

##### TLS settings ######
# see https://thomas-leister.de/erweiterte-postfix-ssltls-konfiguration-verschluesselte-server-server-verbindungen/

tls_ssl_options = NO_COMPRESSION
tls_high_cipherlist=EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA

### outgoing connections ###
smtp_tls_security_level=may
smtp_tls_cert_file=/etc/ssl/certs/postfix.pem
smtp_tls_key_file=/etc/ssl/private/postfix.key
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_bind_address = 95.130.22.100
smtp_bind_address6 = 2a02:2940:0:c007::100

### incoming connections ###
smtpd_sasl_security_options = noanonymous
smtpd_use_tls=yes
smtpd_tls_security_level=may
smtpd_tls_cert_file=/etc/ssl/certs/postfix.pem   # The SSL certificates of the web server are used here!
smtpd_tls_key_file=/etc/ssl/private/postfix.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_mandatory_ciphers = high
smtpd_tls_exclude_ciphers = ECDHE-RSA-RC4-SHA
smtpd_tls_mandatory_exclude_ciphers = ECDHE-RSA-RC4-SHA

# Allow 64 MB message size
message_size_limit = 67108864

# email accounts (hosting)
virtual_alias_domains = s2survey.de, www.s2survey.de, s2survey.com, www.s2survey.com
virtual_alias_maps = hash:/etc/postfix/virtual

# sender aliases
sender_canonical_maps=hash:/etc/postfix/senders

# Limit connections per server (default 20)
smtp_destination_concurrency_limit = 3

# Stop retries after 36 hours (to quickly delete problem-mails)
bounce_queue_lifetime = 24h
maximal_queue_lifetime = 36h

### Basic settings ###
myhostname = s2survey.net
mydestination = $myhostname, mail.s2survey.net, www.s2survey.net, soscisurvey2, localhost.localdomain, localhost
# inet_protocols = ipv4
inet_protocols = all
</file>

<file conf /etc/postfix/senders>
root      info@s2survey.net
www-data  info@s2survey.net
</file>

<file conf /etc/postfix/virtual>
root                         info@soscisurvey.de
webmaster                    info@soscisurvey.de
postmaster                   info@soscisurvey.de
root@mail.s2survey.net       info@soscisurvey.de

mailer@s2survey.net          mailer
survey@s2survey.net          mailer
noreply@soscisurvey.de       mailer

info@s2survey.net            info@soscisurvey.de
info@s2survey.com            info@soscisurvey.de
info@s2survey.de             info@soscisurvey.de

hostmaster@s2survey.net      info@soscisurvey.de
hostmaster@www.s2survey.net  info@soscisurvey.de
hostmaster@s2survey.com      info@soscisurvey.de
hostmaster@www.s2survey.com  info@soscisurvey.de
hostmaster@s2survey.de       info@soscisurvey.de
hostmaster@www.s2survey.de   info@soscisurvey.de
webmaster@s2survey.net       info@soscisurvey.de
webmaster@www.s2survey.net   info@soscisurvey.de
webmaster@s2survey.com       info@soscisurvey.de
webmaster@www.s2survey.com   info@soscisurvey.de
webmaster@s2survey.de        info@soscisurvey.de
webmaster@www.s2survey.de    info@soscisurvey.de
postmaster@s2survey.net      info@soscisurvey.de
postmaster@www.s2survey.net  info@soscisurvey.de
postmaster@s2survey.com      info@soscisurvey.de
postmaster@www.s2survey.com  info@soscisurvey.de
postmaster@s2survey.de       info@soscisurvey.de
postmaster@www.s2survey.de   info@soscisurvey.de

noreply@s2survey.net         info@soscisurvey.de
wiki@s2survey.net            info@soscisurvey.de
root@s2survey.net            info@soscisurvey.de
www-data@s2survey.net        info@soscisurvey.de
</file>

For the return mails we also need a user for the mailbox with password.

<code bash>
sudo adduser mailer --shell /sbin/nologin
</code>

Finally, set up DKIM so that mails from @s2survey.net don't get stuck anywhere.

For this the following instruction: [[https://kofler.info/dkim-konfiguration-fuer-postfix/|DKIM für Postfix]].

<code bash>
sudo mkdir /etc/opendkim
sudo mkdir /etc/opendkim/keys
sudo chown -R opendkim:opendkim /etc/opendkim
sudo chmod go-rw /etc/opendkim/keys
</code>

<file conf /etc/opendkim.conf>
UserID                  opendkim:opendkim
UMask                   002
PidFile                 /var/run/opendkim/opendkim.pid
Socket                  inet:8892@localhost
AutoRestart             yes
AutoRestartRate         10/1h
# Später weniger Logging
Syslog                  yes
SyslogSuccess           yes
LogWhy                  yes
# Nicht zu streng sein
Canonicalization        relaxed/simple
# interne Mails nicht mit OpenDKIM verarbeiten
ExternalIgnoreList      refile:/etc/opendkim/trusted
InternalHosts           refile:/etc/opendkim/trusted
# Keys pro Domain
# (refile: für Dateien mit regulären Ausdrücke)        
SigningTable            refile:/etc/opendkim/signing.table
KeyTable                /etc/opendkim/key.table       
# diesen Signatur-Algorithmus verwenden
SignatureAlgorithm      rsa-sha256
# Always oversign From (sign using actual From and a null From to prevent
# malicious signatures header fields (From and/or others)
OversignHeaders         From
</file>

<file conf /etc/opendkim/trusted>
127.0.0.1
::1
localhost
soscisurvey2
s2survey
s2survey.net
</file>

<file conf /etc/opendkim/signing.table>
# Nur Mails @s2survey.net signieren
*@s2survey.net s2survey
</file>

<file conf /etc/opendkim/key.table>
s2survey s2survey.net:2019a:/etc/opendkim/keys/2019a.private
</file>

<code bash>
sudo su
cd /etc/opendkim/keys
opendkim-genkey -d s2survey.net -b 3072 -r -s 2019a
chown opendkim:opendkim *
less 2019a.txt
</code>

Create a DNS record for s2survey.net (e.g. ''2019a._domainkey.s2survey.net''), according to the contents of the ''2019a.txt'' file. Everything between the brackets goes into the TXT entry, but quotes, spaces and line breaks are removed.

<code bash>
exit
sudo systemctl restart opendkim
opendkim-testkey -d s2survey.net -s 2019a -vvv
</code>

<file conf /etc/postfix/main.cf>
# OpenDKIM
milter_default_action = accept   
milter_protocol   = 6              
smtpd_milters     = inet:localhost:8892
non_smtpd_milters = inet:localhost:8892
</file>


==== Dovecot ====

<file conf /etc/dovecot/conf.d/10-auth.conf>
disable_plaintext_auth = yes
</file>

Test the connection

<code bash>
echo "Subject: E-Mail-Test" | sendmail -v mailer
echo "Subject: E-Mail-Test" | sendmail -v info@soscisurvey.de
printf "From:info@soscisurvey.de\nSubject: E-Mail-Test" | sendmail -v info@soscisurvey.de
</code>

<code bash>
openssl s_client -crlf -connect localhost:993

A1 login mailer [PASSWORD_MAILER]
n namespace
A status INBOX (messages)
g21 SELECT "INBOX"
f fetch 1:1 (BODY[HEADER.FIELDS (Subject)])
</code>




===== Webserver (nginx) ======

s. auch https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-with-http-2-support-on-ubuntu-16-04

==== SSL ====

Generally configured in ''nginx''. However, make sure that the keys are available under ''/var/www/s2survey/ssl'' and that the correct processes can access them.

<code bash>
sudo chown -R root:ssl-cert /var/www/ssl
</code>

===== Reload I =====

<code bash>
sudo systemctl stop php7.4-fpm mysql nginx sshd
sudo systemctl start php7.4-fpm mysql nginx sshd
</code>


===== Install SoSci Survey =====

  * Copy files
  * Customize file permissions
  * Initialize database ''admin/install.php''
  * Check correct function
  * Test serial mail dispatch

<code bash>
cd /var/www/s2survey/html/

sudo chown -R www-ftp:www-data admin images inc lib layout modules plugins script templates
sudo chmod -R 750 admin images inc lib layout modules plugins script templates
# system wird erst von der Installationsroutine angelegt
# sudo chown -R www-data:www-ftp system
# sudo chmod -R 770 system

sudo chown -R www-ftp:nginx admin images inc lib layout modules plugins script templates
sudo chmod -R 750 admin images inc lib layout modules plugins script templates
</code>

The directories ''files'' and ''view'' should automatically get appropriate permissions when SoSci Survey creates them (''www-data''). Controll again at ''system''.


==== SoSci Survey ====

Two scripts (include ''www-data''), may not be copied automatically when transferring the content (permission denied).

<code bash>
sudo mkdir /var/www/s2survey/script
sudo vim /var/www/s2survey/script/crontask.sh
sudo vim /var/www/s2survey/script/cronjob.sh
sudo chown www-data:www-data /var/www/s2survey/script/crontask.sh
sudo chown www-data:www-data /var/www/s2survey/script/cronjob.sh
sudo chmod u+x /var/www/s2survey/script/crontask.sh
sudo chmod u+x /var/www/s2survey/script/cronjob.sh

sudo chown www-data:www-data /var/www/s2survey/script
sudo chown www-data:www-data /var/www/s2survey/log
sudo chmod 700 /var/www/s2survey/script
sudo chmod 700 /var/www/s2survey/log
</code>

<file bash /var/www/s2survey/script/crontask.sh>
#!/bin/bash
cd /var/www/s2survey/html/admin/
/usr/bin/php /var/www/s2survey/html/admin/crontask.php password=123456
</file>

<file bash /var/www/s2survey/script/cronjob.sh>
#!/bin/bash
cd /var/www/s2survey/html/admin/
/usr/bin/php /var/www/s2survey/html/admin/cronjob.php password=123456
</file>




===== Cronjobs =====

If the system clock runs UTC, then accordingly everything 2 hours earlier.

<code>
sudo crontab -e

# m     h       dom     mon     dow     command
@reboot                                 /root/mailinfo.reboot.sh
15      00      *       *       *       /etc/cron.daily/chkrootkit > /var/log/chkrootkit/daily.log
30      00      *       *       *       /etc/cron.daily/rkhunter
00      01      *       *       *       /root/backup/backup-daily.sh
00      02      *       *       sun     /root/backup/backup-weekly.sh
30      04      *       *       *       /root/mailinfo.update.sh
00      05      *       *       sun     /usr/bin/certbot renew --post-hook "service nginx reload"
55      05      *       *       sun     /sbin/reboot
</code>

<code>
sudo crontab -e -uwww-data

# m     h       dom     mon     dow     command
*/1     *       *       *       *       /var/www/s2survey/script/crontask.sh >/dev/null 2>&1
0      3       *       *       *       /var/www/s2survey/script/cronjob.sh
</code>


===== Monitoring =====


==== Munin ====

<code bash>
sudo ln -s /usr/share/munin/plugins/nginx_request /etc/munin/plugins/
sudo ln -s /usr/share/munin/plugins/nginx_status /etc/munin/plugins/
sudo ln -s /usr/share/munin/plugins/http_loadtime /etc/munin/plugins/
sudo ln -s /usr/share/munin/plugins/iostat /etc/munin/plugins/
sudo ln -s /usr/share/munin/plugins/iostat_ios /etc/munin/plugins/
sudo ln -s /usr/share/munin/plugins/postfix_mailqueue /etc/munin/plugins/
sudo ln -s /usr/share/munin/plugins/postfix_mailvolume /etc/munin/plugins/
</code>

Another extra plugin for PHP-FPM

<code bash>
cd /usr/share/munin/plugins/
sudo wget -O php-fpm https://raw.github.com/MorbZ/munin-php-fpm/master/php-fpm.php
sudo chmod +x php-fpm
sudo ln -s /usr/share/munin/plugins/php-fpm /etc/munin/plugins/php-fpm-memory
sudo ln -s /usr/share/munin/plugins/php-fpm /etc/munin/plugins/php-fpm-cpu
sudo ln -s /usr/share/munin/plugins/php-fpm /etc/munin/plugins/php-fpm-count
sudo ln -s /usr/share/munin/plugins/php-fpm /etc/munin/plugins/php-fpm-time
sudo service munin-node restart
</code>


<code bash>
sudo vim /etc/munin/munin.conf

> [localhost.localdomain]
< [s2survey.net]
</code>

<code bash>
sudo htpasswd -c /etc/munin/munin-htpasswd admin
</code>

Zum Testen eines PlugIns.

<code bash>
sudo munin-run nginx_status
</code>


==== Monit ====

<code bash>
# PHP-Service herausfinden
systemctl list-unit-files | grep enabled
sudo vim /etc/monit/conf.d/s2survey.conf
sudo vim /etc/monit/conf.d/mysql
sudo vim /etc/monit/conf.d/nginx
sudo vim /etc/monit/conf.d/php-fpm
sudo vim /etc/monit/conf.d/fail2ban
sudo vim /etc/monit/php-fpm.stop.sh
sudo vim /etc/monit/conf.d/system
sudo vim /etc/monit/conf.d/sosci
sudo vim /etc/monit/webserver.stop.sh
sudo vim /etc/monit/webserver.start.sh
sudo vim /etc/monit/webserver.restart.sh
# Nur wenn relevant
sudo vim /etc/monit/conf.d/postfix
</code>

<file conf /etc/monit/conf.d/s2survey.conf>
set daemon 20 START DELAY 120
set mail-format { from: monit@s2survey.net }
set httpd
        port 2812
        allow admin:"XXXXXXXXXXX"
</file>

The name of the PID file must first be determined in MySQL:

<code mysql>
show variables like '%pid%';
</code>

<file conf /etc/monit/conf.d/mysql>
check process mysqld with pidfile /var/run/mysqld/mysqld.pid
    start program = "/usr/sbin/service mysql start"
    stop program = "/usr/sbin/service mysql stop"
    if failed host 127.0.0.1 port 3306
        for 3 cycles
        then restart
    if 3 restarts within 20 cycles
        then timeout
</file>

<file conf /etc/monit/conf.d/nginx>
check process nginx with pidfile /var/run/nginx.pid
   start program = "/usr/sbin/service nginx start" with timeout 60 seconds
   stop program  = "/usr/sbin/service nginx stop"
   if failed port 80 for 2 cycles
     then restart
   if cpu > 50% for 2 cycles
     then alert
</file>

<file conf /etc/monit/conf.d/php-fpm>
check process php-fpm with pidfile /var/run/php/php7.4-fpm.pid
        group php-www
        start program = "/bin/systemctl start php7.4-fpm"
        stop program = "/etc/monit/php-fpm.stop.sh"
        if failed unixsocket /var/run/php/php-fpm.sock
                then alert
        if failed unixsocket /var/run/php/php-fpm.sock
                for 2 times within 3 cycles
                then restart
        if failed unixsocket /var/run/php/php-fpm.sock
                for 5 cycles
                then exec "/sbin/reboot"
        if 5 restarts within 8 cycles
                then alert
</file>

<file conf /etc/monit/conf.d/fail2ban>
check process fail2ban with pidfile /var/run/fail2ban/fail2ban.pid
  group services
  start program = "/etc/init.d/fail2ban force-start"
  stop  program = "/etc/init.d/fail2ban stop || :"
  if failed unixsocket /var/run/fail2ban/fail2ban.sock then restart
  if 5 restarts within 5 cycles then timeout

check file fail2ban_log with path /var/log/fail2ban.log
    if match "ERROR|WARNING" then alert
</file>

<file bash /etc/monit/php-fpm.stop.sh>
#!/bin/bash

# Stop php-fpm
/bin/systemctl stop php7.4-fpm

# Kill any further instances of PHP-FPM
/usr/bin/pkill php7.4-fpm
</file>

<file conf /etc/monit/conf.d/system>
check system [DOMAINNAME]
        group resources
        if memory usage > 75% for 5 cycles then alert
                else if passed within 5 cycles then alert
        if loadavg (15min) is greater than 1.5 for 5 cycles then alert
                else if passed within 5 cycles then alert
        if cpu usage (user) > 75% for 2 cycles then alert
        if cpu usage (system) > 25% for 2 cycles then alert
        if cpu usage (wait) > 50% for 2 cycles then alert

check filesystem rootfs with path /
        if space usage > 75% then alert
check device bootfs with path /boot
        if space usage > 75% then alert
</file>

<file conf /etc/monit/conf.d/sosci>
# Website must be accessible
CHECK HOST sosci WITH ADDRESS 95.130.22.100
  START PROGRAM = "/etc/monit/webserver.start.sh"
  STOP PROGRAM  = "/etc/monit/webserver.stop.sh"
  # IF NOT EXIST THEN ALERT
   # if failed (url https://www.soscisurvey.de/example/?debug&password=demo and content == 'onlineFragebogen' and timeout 20 seconds)
   #   then alert
  IF FAILED (url https://s2survey.net/admin/?stay and content == 'SoSci Survey' and timeout 20 seconds)
    FOR 2 CYCLES
    THEN RESTART
  IF FAILED (url https://s2survey.net/admin/?stay and content == 'SoSci Survey' and timeout 20 seconds)
    FOR 4 CYCLES
    THEN EXEC "/sbin/reboot"
    
# Crontask must run every minute
CHECK FILE crontask WITH PATH /var/www/s2survey/html/system/logfiles/crontask.log
    IF DOES NOT EXIST THEN ALERT
    IF timestamp > 20 minutes THEN ALERT

# Cronjob must run daily
CHECK FILE cronjob WITH PATH /var/www/s2survey/html/system/logfiles/cronjob.log
    IF DOES NOT EXIST THEN ALERT
    IF timestamp > 25 hours THEN ALERT
</file>

<file bash /etc/monit/webserver.stop.sh>
#!/bin/bash

# Stop php-fpm
/etc/monit/php-fpm.stop.sh

# Stop nginx
/bin/systemctl stop nginx
</file>

<file bash /etc/monit/webserver.start.sh>
#!/bin/bash
/bin/systemctl start php7.4-fpm
/bin/systemctl start nginx
</file>

<file bash /etc/monit/webserver.restart.sh>
#!/bin/bash
/etc/monit/webserver.stop.sh
/etc/monit/webserver.start.sh
</file>

<file conf /etc/monit/conf.d/postfix>
CHECK PROCESS postfix WITH PIDFILE /var/spool/postfix/pid/master.pid
    start program = "/usr/sbin/service postfix start"
    stop program  = "/usr/sbin/service postfix stop"
    if cpu > 60% for 2 cycles then alert
    if cpu > 80% for 5 cycles then restart
    if failed host localhost port 25 type tcp protocol smtp
       with timeout 15 seconds
      then alert
    if 3 restarts within 5 cycles then timeout
</file>


<code bash>
sudo chmod u+x /etc/monit/php-fpm.stop.sh
sudo chmod u+x /etc/monit/webserver.stop.sh
sudo chmod u+x /etc/monit/webserver.start.sh
sudo chmod u+x /etc/monit/webserver.restart.sh
sudo monit -t
</code>


===== Reload II =====

<code bash>
sudo systemctl reload monit
sudo systemctl restart munin-node unattended-upgrades
</code>


===== Tests =====

==== Create Projekt ====

Dito

==== E-Mail ====

Some adjustments are required for the outgoing emails to get through the spam filter. To test with [[https://www.mail-tester.com/|mail tester]].

You can check the TLS configuration with [[https://www.checktls.com/|checktls.com]].

Returns over https://[SOSCI_URL]/admin/index.php?o=Receiver&a=bounces

Newsletter configuration: https://www.mail-tester.com/?lang=de

==== Backups ====

<code bash>
sudo su
duplicity collection-status file:///mnt/backup/soscisurvey2020/database
duplicity list-current-files file:///mnt/backup/soscisurvey2020/database
duplicity restore -t 2020-12-31T12:00:00-01:00 file:///mnt/backup/soscisurvey2020/database /var/backup/restore/
</code>


==== Other ====

  * Owncloud
  * Manual
  * SMS-Sending


===== Move Server =====

  * Disable Monit on both servers
  * Blanc SoSci-Cache
  * Lock Server
  * Export database
  * Copy data
  * Import database 
  * Set database access data in SoSci Survey
  * At www.soscisurvey.de add the ''sites-available'' (home page, instructions, owncloud).
  * Change IP-Adress
    * Remove primary address on old system
    * Set primary address on the new system (keep old one as secondary address for now)
    * When everything is running: Swap secondary address
  * delete transfer data (SQL)


===== User for File Move =====

If the keyfile login should remain active, it is a good idea to create a new user on the old server.

For this we first need a new key pair on the new server

<code bash>
ssh-keygen -t rsa -b 4096 -C "transfer"
# Filename: .ssh/transfer
</code>

Then the new user on the old server.

<code bash>
sudo su
adduser transfer --gecos ""
# Passwort eingeben oder auch nicht
cd /home/transfer
mkdir .ssh && chown transfer .ssh && chmod 700 .ssh
vim .ssh/authorized_keys2
# Inhalte aus transfer.pub vom anderen Server rüberkopieren
</code>

Finally, test the access.

<code bash>
ssh transfer@141.84.43.202 -i ~/.ssh/transfer.key
</code>

And then of course copy files - via ''/home/transfer/''..

<code bash>
scp  -i /home/leiner/.ssh/transfer.key transfer@141.84.43.202:/home/transfer/soscipanel* /var/www/ssl/
</code>

==== Copy Contents ====

  * Files /root/backup/*.sh
  * Files under /var/www/
    * SoSci Survey
    * SSL-Certificates (if available)

<code bash>
# In advance: Clear working cache, clear write cache

# Then copy the files
sudo rsync -av -e ssh --delete sosci@s2survey.net:/var/www/s2survey/html/ /var/www/s2survey/html/
sudo rsync -av -e ssh --delete sosci@s2survey.net:/var/www/s2survey/log/ /var/www/s2survey/log/
sudo rsync -av -e ssh --delete sosci@95.130.22.99:/var/www/shibboleth /var/www/

sudo chown -R www-data:www-data /var/www/s2survey/html
sudo chown -R www-data:www-data /var/www/s2survey/html/shop/system/keyring
sudo chmod 700 /var/www/s2survey/html/system/keyring
sudo chmod 700 /var/www/s2survey/html/shop/system/keyring
sudo chmod 600 /var/www/s2survey/html/system/keyring/*
sudo chmod 600 /var/www/s2survey/html/shop/system/keyring/*
sudo chown -R www-data:www-ftp /var/www/s2survey/html/admin
sudo chown -R www-data:www-ftp /var/www/s2survey/html/inc
sudo chown -R www-data:www-ftp /var/www/s2survey/html/lib
sudo chown -R www-data:www-ftp /var/www/s2survey/html/images
sudo chown -R www-data:www-ftp /var/www/s2survey/html/internal
sudo chown -R www-data:www-ftp /var/www/s2survey/html/layout
sudo chown -R www-data:www-ftp /var/www/s2survey/html/modules
sudo chown -R www-data:www-ftp /var/www/s2survey/html/plugins
sudo chown -R www-data:www-ftp /var/www/s2survey/html/script
sudo chown -R www-data:www-ftp /var/www/s2survey/html/system
sudo chown -R www-data:www-ftp /var/www/s2survey/html/templates

sudo chown -R www-data:root /var/www/s2survey/log
</code>

Possibly also copy /etc/letsencrypt. Then pass it to the user ''root'' again.

<code bash>
sudo mysqldump -uroot -p --databases s2survey --quick --single-transaction | gzip -9 > /home/sosci/transfer-s2survey.sql.gz
</code>

On the other server, get the data and import it.

<code bash>
scp sosci@s2survey.net:~/transfer* ~/
gunzip -c ~/transfer-s2survey.sql.gz | mysql -u master -p s2survey
</code>

  * Test RTR-Mobile
  * Test Owncloud
  * Test shop + lizenz activation


==== Change IP-Adress ====

To change the IP-Adress see [[https://wiki.ubuntuusers.de/Netplan/|netplan]] and [[https://www.linux.com/learn/intro-to-linux/2018/9/how-use-netplan-network-configuration-tool-linux|linux.com]].

It may be useful to switch one IP address first and then the other.

<file conf /etc/netplan/01-netcfg.yaml>
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
  version: 2
  renderer: networkd
  ethernets:
          ens160:
                  dhcp4: no
                  dhcp6: no
                  addresses: [95.130.22.100/28,95.130.22.101/28]
                  gateway4: 95.130.22.97
                  nameservers:
                          addresses: [95.130.16.100,95.130.16.121]
</file>

The configuration can then be test loaded as follows:

<code>
sudo netplan try
</code>

After that you have to make sure that the routers and switches update the ARP cache. For this you need the name of the interface (here/top is ens160) and...
<code bash>
sudo apt install iputils-arping

arping -U -I ens160 95.130.22.98
</code>


==== At Least ====

  * Test mailsending
  * Test returns
  * Check connected domains
  * Update der SSL-Zertifikate über Let's Encrypt testen
  * Logrotation for ''nginx'' access logs

<code bash>
sudo certbot renew --force-renewal
</code>


==== Owncloud ====

  * Disable client on all PCs/laptops
  * Enable log file rotation on the new server
  * Check file permissions after the move
  * Cronjob for maintenance ''sudo crontab -u www-data -e''
  
<code conf>
*/15 *    *     *   *    php /var/www/s2survey/html/owc/occ system:cron
</code>


===== Switch on Domain =====

<code bash>
sudo su
cd /var/www/s2survey/html
vim VERZEICHNIS/index.php
# chdir() ergänzen (s. unten)

ln -s VERZEICHNIS _DOMAINNAME
vim /etc/nginx/sites-available/02.customers.conf
# (1) Domain oben im HTTP:80 ergänzen
# (2) Neuer Eintrag für die Domain (sonst müssten sie ein Zertifikat teilen)
#     => Aber erstmal mit Zertifikat von anderer Domain, sonst schlägt der Config-Test fehl
nginx -t
sudo systemctl reload nginx

certbot certonly --webroot --debug-challenges --dry-run -w /var/www/s2survey/certbot -d DOMAINNAME
certbot certonly --webroot -w /var/www/s2survey/certbot -d DOMAINNAME
</code>

<code php>
<?PHP
// change to the correct working directory
chdir('../PROJECT_DIR/');
// Start or continue interview
require('../inc/Interviewer.php');
Interviewer::run(PROJECT_ID);
?>
</code>

<code conf>
server {
        listen 95.130.22.101:443 ssl http2;
        listen [2a02:2940:0:c007::101]:443 ssl http2;
        server_name DOMAINNAME;

        root /var/www/s2survey/html/_$host;
        index index.php index.html index.htm;
        client_max_body_size    64M;

        access_log /var/www/s2survey/log/n-access-custom.log noagent;
        error_log /var/www/s2survey/log/n-error-custom.log;

        ssl_certificate_key /var/www/ssl/s2survey.2019.key;
        ssl_certificate /var/www/ssl/s2survey.2019.crt;

        # ssl_certificate_key /etc/letsencrypt/live/DOMAINNAME/privkey.pem; # managed by Certbot
        # ssl_certificate /etc/letsencrypt/live/DOMAINNAME/fullchain.pem; # managed by Certbot


        location ~ /\.ht {
                deny all;
                return 403;
        }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        location ~ \.php$ {
                limit_req zone=php burst=50;
                try_files $uri =404;

                fastcgi_intercept_errors      on;
                fastcgi_next_upstream         error timeout http_500 http_503;
                fastcgi_next_upstream_timeout 10s;
                fastcgi_next_upstream_tries   2;
                fastcgi_pass sosci;

                fastcgi_index index.php;
                fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
                include fastcgi_params;
                # Limitations for survey projects
                fastcgi_param PHP_VALUE "
                        max_execution_time = 5
                        memory_limit = 64M
                        max_file_uploads = 5
                        post_max_size = 16M
                ";
        }

        # Cache management
        location ~* \.(?:ico|css|js|gif|jpg|jpeg|png|flv|pdf|swf)$ {
                expires 1d;
                add_header Vary Accept-Encoding;
                add_header X-Content-Type-Options nosniff;
                # access_log off;
        }

        # Anything else
        location ~ / {
                # First attempt to serve request as file, then
                # as directory, then fall back to index.html
                try_files $uri $uri/ =404;
        }
}
</code>


===== Other =====


==== CSR for SSL =====

<code bash>
cd /var/www/ssl

# Create key:
sudo su
openssl genrsa 4096 > s2survey.2019.key
# Certificate Request
openssl req -new -key s2survey.2019.key -out s2survey.2019.csr

Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Bayern
Locality Name (eg, city) []:Muenchen
Organization Name (eg, company) [Internet Widgits Pty Ltd]:SoSci Survey GmbH
Organizational Unit Name (eg, section) []:Survey Hosting
Common Name (e.g. server FQDN or YOUR name) []:s2survey.net
Email Address []:info@soscisurvey.de

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
</code>

2 sets of SSL certificates are used:

  * Set 1: Organization Validation (OV)
    * www.soscisurvey.de (primäre Domain)
    * s2survey.net (Pro-Server Survey)
    * admin.s2survey.net (Pro-Server Verwaltung)
  * Set 2: Secure Site (für Weiterleitungen) -> da wäre auch Let's Encrypt denkbar
    * soscisurvey.de (Falls man die Adresse ohne www einträgt)
    * admin.soscisurvey.de (Administration, explizit nicht premium abgesichert)
    * www.soscisurvey.net (evtl. überflüssig)
    * s2survey.com
    * s2survey.de
  * Set 3 von Subdomains ohne SSL oder via Let's Encrypt
    * www.soscisurvey.com
    * www.s2survey.net (falls jemand das www voranstellt)

===== IPv6-Adress =====

The IP address is set either under ''/etc/network/interfaces'' oder via NetworkManager (''nmtui'') or via ''netplan''.

<code bash>
vim /etc/netplan/...
</code>
  
<file netplan /etc/netplan/00-installer-config.yaml>
network:
  ethernets:
    ens160:
      addresses:
      - 95.130.22.103/28
      - 2a02:2940:0:c007::103/64
      gateway4: 95.130.22.97
      gateway6: 2a02:2940:0:c007::1
      nameservers:
        addresses:
        - 95.130.16.100
        - 95.130.16.121
  version: 2
</file>

It is agreed with Partnergate that the IPv6 ''2a02:2940:0:c007::102/64'' is used for the server .102 and so on.

Der Gateway ist ''2a02:2940:0:c007::1''.

<code bash>
/sbin/ip -6 route show dev eth0
</code>

**Important:** Before enabling IPv6, also adjust the firewall accordingly.


==== NetworkManager ====

Some commands for the NetworkManager:

  * ''nmcli device'' und ''nmcli connection''
  * ''ip --family inet6 addr show dev eth0 scope global''
  * ''nmcli device show eth0''
  * ''nmcli connection reload''
  * ''nmtui''
  * ''%%elinks http://www.wieistmeineip.de%%''
  * ''ping6 ipv6.google.com''
  * ''/etc/NetworkManager/system-connections/Wired connection 1''
  * ''/etc/NetworkManager/conf.d/99-local.conf'' ([[https://vk5tu.livejournal.com/57160.html|Activating IPv6 stable privacy addressing from RFC7217 ]])

<file ini /etc/NetworkManager/conf.d/99-local.conf>
[connection]
ipv6.ip6-privacy=0
ipv6.addr-gen-mode=stable-privacy
</file>

<file ini /etc/NetworkManager/system-connections/Wired connection 1>
[connection]
id=Wired connection 1
uuid=92b0a2d7-d185-4bf7-b3e9-8606a0e08355
type=ethernet
permissions=
timestamp=1574712975

[ethernet]
mac-address-blacklist=

[ipv4]
address1=95.130.22.102/28,95.130.22.97
dns=95.130.16.100;95.130.16.121;
dns-search=
method=manual

[ipv6]
addr-gen-mode=stable-privacy
address1=2a02:2940:0:c007::102/64,2a02:2940:0:c007::1
dns-search=
ip6-privacy=0
method=manual
</file>


===== M/Monit =====

On mail/secure runs M/Monit to collect status messages from own and managed servers.

Collect URL ''https://mmonit.com/download/'' (linux-x64)

<code bash>
cd ~
wget https://mmonit.com/dist/mmonit-3.7.3-linux-x64.tar.gz
tar -xvf mmonit-*
sudo mv mmonit-3.7.3 /opt/mmonit
cd /opt/mmonit/
./bin/mmonit
sudo vim /etc/nginx/sites-available/mail.soscisurvey.de
</code>

<code>
    location /mmonit/ {
        proxy_ignore_client_abort on;
        proxy_pass http://127.0.0.1:8080/;
    }
</code>

<code bash>
sudo nginx -t
sudo systemctl reload nginx
</code>

Point your Browser to the [[https://mail.soscisurvey.de/mmonit/|host where mmonit is installed]], and login as user "admin" with password "swordfish".

There under Admin/Users change the passwords:

  * ''admin'' for the management,
  * ''monit'' for the other servers to send data.

Furthermore Admin/Alerts/Message Settings -> monit@soscisurvey.de

<code bash>
sudo vim /opt/mmonit/docroot/WEB-INF/web.xml
</code>

So that ''monit'' can only send data and not retrieve info....

<code conf>
            <role-name>admin</role-name>
            <!-- <role-name>*</role-name> -->
</code>

The license code is located (at the very bottom) in ''/opt/mmonit/conf/server.xml''.

Set up auto start loud [[https://mmonit.com/wiki/MMonit/Setup#autolaunch|offizieller Anleitung]].

<code bash>
sudo vim /etc/monit/conf.d/monit.conf
</code>

<code>
check process mmonit with pidfile /opt/mmonit/logs/mmonit.pid
   start program = "/opt/mmonit/bin/mmonit"
   stop program = "/opt/mmonit/bin/mmonit stop"
</code>

Add to the nodes/hosts...

<code bash>
sudo vim /etc/monit/conf.d/s2survey.conf
sudo systemctl reload monit
</code>

<code>
set eventqueue basedir /var/monit/ slots 1000
set mmonit https://monit:[PASSWORD]@mail.soscisurvey.de/mmonit/collector
</code>


===== SimpleSAML =====

The configuration of the environment variable for ''config'' is performed in ''/etc/php/7.4/fpm/pool.d/primary.conf''.

==== Update ====

https://simplesamlphp.org/download -> URL
https://simplesamlphp.org/download?latest

Upgrade according to [[https://simplesamlphp.org/docs/stable/simplesamlphp-install#section_3|Anleitung]] with minor optimizations.

<code bash>
cd /var/www/shibboleth
sudo wget https://simplesamlphp.org/download?latest
sudo tar -xvf "download?latest"
sudo rm "download?latest"
cd simplesamlphp-1.NEUESTE
sudo touch modules/cron/enable
sudo touch modules/metarefresh/enable

# The configuration is now directly in /var/www/shibboleth/config/
# cd /var/simplesamlphp-x.y.z
# sudo rm -rf config metadata
# sudo cp -rv ../simplesaml/config config
# sudo cp -rv ../simplesaml/metadata metadata
# sudo cp -rv ../simplesaml/cert cert

# Synchronize changes
cd /var/www/shibboleth
diff -u simplesamlphp-x.y.z/config-templates/config.php config/config.php

sudo rm simplesaml
sudo ln -s simplesamlphp-1.NEUESTE simplesaml

</code>


===== Live-Monitoring =====

Split-Screen via ''tmux''.

''<strg>+B'' -> ''"'' für horizontale Teilung
''<strg>+B'' -> ''%'' für vertikale Teilung
''<strg>+B'' -> Pfeiltaste zum Umschalten zwischen den Feldern

<code bash>
sudo apachetop -f /var/www/s2survey/log/n-access.log
elinks  http://localhost/status.primary?full
top
</code>

Also, keep an eye on Munin.


===== Bugfixes =====

>  warning: btree:/var/lib/postfix/smtp_scache is unavailable

<code bash>
sudo systemctl stop postfix
sudo rm -f /var/lib/postfix/smtpd_scache.db
sudo systemctl start postfix
</code>

> warning: btree:/var/lib/postfix/smtp_scache is unavailable. open database /var/lib/postfix/smtp_scache.db: File exists

<code bash>
sudo systemctl stop postfix
sudo rm -f /var/lib/postfix/__db.smtp_scache.db
sudo systemctl start postfix
</code>

... or whatever old files are there and block the system.


===== Test System Setup =====

A test system was set up for a penetration test in 04/2021. Here are the key points.

Partnergate set up a lon of the VM and changed the IP address:

  * Partnergate needs a sudo-Account
  * After cloning, the mount ''/mnt/backup'' was removed.
  * Only the IPv4 address was changed, the server did not get IPv6
  * Monit must be deactivated first ''sudo systemctl stop monit'', so that the server does not restart constantly.
  * Next, the cronjob password in ''system/config.php'' should be changed so that no more serial emails are sent.

A new backup is first made of the production system.

The following steps are necessary for the test system:

  * Setting up a new license on the distribution server
  * Change server address via ''admin/install.php'' (double login may be necessary) + enter new license
  * Delete a project -- and make sure you're on the right server
  * Empty the Tables ''s2_activities'', ''s2_log'', ''s2_scheduled'' und ''s2_messages''
  * Deletion of all projects and users, except for a handful of test projects
    * ''DELETE FROM s2_projects WHERE id NOT IN (1, 2, 102)''
    * ''DELETE FROM s2_users WHERE id NOT IN (1, 2, 3, 55)''
    * Check tables again for mail addresses etc.
  * Restrict Monit on the test system, disable reboot
    * Delete keywords (2x) in ''/etc/monit/conf.d/s2survey.conf''
    * Delete of ''/etc/monit/conf.d/sosci''
  * Delete all directories from  ''/root/.duply''
  * ''sudo crontab -e -uroot'' -> deactivate Backup, Reboot and Certbot
  * ''sudo crontab -e -uwww-data'' -> deactivate the nightly Cronjob 
  * Change IP address to ''sites-available/02.customers'' and remove foreign domains. Enter the testing domain in the port 80 part, so that the registration via Let's Encrypt works.
  * Create new certificate (see above) via Certbot
  * Create user account for the tester, possibly also transfer admin account.
  * Change keywords
    * Unix-user, see ''/etc/shadow'' -- ''mailer'' also register in SoSci Survey
    * ''authorized_keys'' for ''sosci'' and swap ''www-ftp'' (to a new key pair) -> PuttyGen
    * MySQL user change all passwords and enter new ''s2survey'' password in SoSci Survey (directly in ''config.php'')
    * Munin: ''sudo htpasswd -c /etc/munin/munin-htpasswd admin''
  * From ''sytem/keyring/'' delete everything except the license server, then go to server settings to rstellen a new key. DEnter this on the distribution server (''keyring'' and ''config.php''). Adjust the ''config.php'' on the test system.
  * Delete password for SMS sending from SoSci configuration.
  * Delete Server Protkolls
    * Logfiles nginx (after that ''sudo systemctl restart nginx'')
    * sosci Logfiles
    * mail.log, danach ''sudo systemctl restart postfix''
    * syslog  ''truncate -s 0 /var/log/syslog''
  * Delete Certbot directories under ''/etc/letsencrypt/archive'', ''/etc/letsencrypt/live'', ''/etc/letsencrypt/renewal'', 